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Efficient Computation of Spectral Bounds 
for Hessian Matrices on Hyperrectangles 
for Global Optimization^ 

Moritz Schulze Darup, Martin Kastsian, Stefan Mross 
and Martin Monnigmanro 



Abstract. We compare two established and a new method for the calculation of spectral 
bounds for Hessian matrices on hyperrectangles by applying them to a large collection of 1522 
objective and constraint functions extracted from benchmark global optimization problems. 
Both the tightness of the spectral bounds and the computational effort are assessed. Specifi- 
cally, we compare eigenvalue bounds obtained with the interval variant of Gcrshgorin's circle 
criterion [2j[6], Hertz and Rohn's [3 US] method for tight bounds of interval matrices, and a 
recently proposed Hessian matrix eigenvalue arithmetic |12] , which deliberately avoids the com- 
putation of interval Hessians. 
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1 Introduction 



We compare a recently proposed method [TU] for the calculation of spectral bounds for Hessian 
\Q , matrices on hyperrectangles to existing ones. We begin with a concise problem statement. Let 

Q\ ' ip : U C R" ->Rbea twice continuously differentiable function on an open set U C R™ and let 

B = \ x_iiX\\ x ■ • • x [a;„,a; n ] be a closed hyperrectangle in U. The problem of interest reads as 
follows. 

'. 

OFind A G R, A G R such that 

A < A < A for all eigenvalues A of all matrices H € Ti(<p, B), 
where H(ip, B) is the set of Hessian matrices of ip on B 
X ■ H(<p,B) = {V 2 <p(x)\xeB}. (2) 

S ■ 

A bound A (resp. A) is called tight if there exists at least one matrix H in the matrix set with 
an eigenvalue A = A (resp. A = A). Note that the bounds A, A in ([1]) may or may not be tight. 

Problem ([I} appears in various applications. It is crucial, for example, to establish the 
convexity of nonlinear functions in nonlinear optimization, since methods for solving nonconvex 
optimization problems are much less efficient than those for their convex counterparts. If (JlJ 
results in X > then cp is convex on the interior of the hyperrectangle B [H [15]. If, in contrast, 
A < results from ([1}, then ip(x) may or may not be convex on B, but 

1 - 

$(x) = (p(x)--\22(x i -Xi)(xi-Xi) (3) 

i=l 
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is a convex function that underestimates f on B and coincides with if at the vertices of B [H |3] . 
Underestimators of this type are employed in nonconvex global optimization to bound the global 
minimum from below. Essentially, B is bisected into smaller and smaller hyperrectangles B in 
these approaches to obtain tighter and tighter convex underestimators. This requires solving 
([1]) repeatedly for different domains B c B but the same function if. As a result, a considerable 
fraction of the total computational time is spent on the calculation of convex underestimators [1] . 
Consequently, fast methods for solving (JIJ are of interest in this field. Problem ([TJ also arises in 
automatic control and systems theory. We refer to [11] for a simple example, where eigenvalue 
bounds for Hessian matrix sets are used to prove the positive or negative invariance of regions 
in the state space of nonlinear dynamical systems. 

Problem (JTJ is commonly solved in two steps: (i) The interval Hessian matrix is calculated, 
(ii) One out of several existing methods that provide bounds on the eigenvalues of symmetric 
interval matrices [21 [Jj 116) is applied. Interval Hessian matrices can efficiently be computed by 
combining interval arithmetics (IA for short; see, e.g., |13j ) and automatic differentiation (see, 
e -g-> [HI IS)- This results in intervals [V 2 ip , V 2 <^y] C R, % = 1, . . . , n, j = 1, . . . , n, such that 



( v VW), 3 6[v^ y ,vV y ] (4) 



for all x € B, where V 2 ^ . = V 2 f .. and V 2 ^- — ^ 2( Pji due to symmetry of V 2 if(x). We refer 
to the set of matrices 

n lA (f, B) = {He M" x " \H tJ e [Vy.^VV,,], H = H T I (5) 

as the interval Hessian of <f on B. After calculating - H IA ((/?, B), the spectral bounds can be 
found by solving the following problem. 



Find A e R, A € K such that 

A < A < A for all eigenvalues A of all matrices H g H lA (if, B). 



ia, ^ (6) 



The calculation of H IA {<f, B) requires 0(n 2 ) N(ip) operations if the forward mode of automatic 
differentiation [5] is used, where N((f) denotes the number of operations needed to evaluate if 
at a point in its domain. With the backward mode of automatic differentiation, this complexity 
can be reduced to 0(n) N((p) [S]. 

There exist a number of approaches to solving ([6]). Assuming the interval Hessian Ti. lA {ip, B) 
is available, the computational complexity of these methods varies between 0{n 2 ) for the interval 
variant of Gershgorin's circle criterion [HH] and 0(2™ n 3 ) for Hertz and Rohn's method [71 116). 
which provides tight spectral bounds for 'H IA (</>,.B) (see Sect. I2.1l and [2.3l for details). However, 
since H((f, B) C "H IA (<p, B), problem (JSJ) is conservative compared to the original problem ([1]). In 
[12] . we introduced a method for solving ([TJ that does not require the interval Hessian H} A (f, B) 
and therefore avoids the conservatism inherent in ([6]). The major advantage of this method is 
the low computational complexity, which was shown to be of order 0(n) N(if) [T2]. Note that 
the total numerical effort of the approaches mentioned before is the sum of the complexity for 
calculating % IA (</?, B) and solving ©. At least 0(n) N(<p) + 0(n 2 ) operations are needed in 
these cases, and typical implementations based on forward mode automatic differentiation and 
Gershgorin's circle criterion require 0(n 2 ) N(<p) + 0(n 2 ) operations. 

It is the purpose of this paper to compare spectral bounds obtained with the recently pro- 
posed method [l2j to those calculated by applying Gershgorin's circle criterion and Hertz and 
Rohn's method to the interval Hessian. Since the motivation for developing a new method was 
the application of Hessian eigenvalue bounds in global optimization, we apply the three com- 
pared methods to a large set of test functions generated from a collection of benchmark global 
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optimization problems. Specifically, we extract 1522 objective and constraint functions from the 
COCONUT collection [T7]. For each function, we randomly generate 100 hyperrectangles in its 
domain and compute the associated lower and upper eigenvalue bounds with the three methods. 
We compare both the resulting spectral bounds and the number of operations required by each 
of the approaches. 

After introducing some notation in the remainder of this section, the three methods are 
summarized in Sect. [2l The central benchmark, which constitutes the main result of the paper, 
is stated in Sect. [3] Finally, conclusions are given in Sect. |U 

Notation. Pairs of lower and upper bounds such as A < A < A are denoted by intervals, 
i.e. A G [A, A] C R, for short. Intervals [a, a] are further abbreviated by [a] :— [a, a]. Interval 
equality [a] — [6] is understood as a — 6 and a = b. Calculations with intervals are carried out 
with standard interval arithmetics rules, which are collected in Fact Q] without proof (see, e.g., 

EE]). 

Fact 1 (basic interval arithmetics ) Let [a] and [b] be intervals and a € [a], b G [b] and eel fc 



arbitrary real numbers. Then 

a + b G [a] + [b] := \a + b,a + b], (7) 

ab G [a] [6] :— [min (a6, a 6, a 6, a 6) , max (ab,ab,ab,ab^]. (8) 

1/6 G l/[fo] := [1/6, 1/6] (9) 

a + c G [a] + c := [a + c,a + c] (10) 

, , f \ca,ca] if c > 
ca G c [a }:= < z' % ~ „ (11) 

[ [ca, coj if c < 0, 



where ^ [6, 6] is assumed (0J). Furthermore, the power of natural numbers m G N, the square 
root, the exponential and the natural logarithm of an interval are defined as follows. 

if a > or m odd 

ifa<0 and m even (12) 
if G [a] and m even 

(13) 

(14) 
(15) 

where a > are assumed in 113\) and H5\) . respectively. 

By a slight abuse of notation we denote both a real interval [x] — [x, x] C R and a hyperrectangle 
[x] = [x,x] = [x l7 xi] x ■ • • x [x n ,x„] C R™, n > 2 by a lower case letter surrounded by brackets. 
As a generalization of Eqs. ([T^ - flTSl) . interval extensions of functions f(x), f : U C M™ —> M., 
n > 1, are denoted by [/([a;])]. We denote gradients and the Hessian matrices of a function 
/ : 1/ C I'M I by V/(x) and V 2 /(x), respectively, if they exist. Whenever V/ V/^ G R, 

i = 1, . . . , n are known, then these bounds define an interval vector denoted by [V/] = [V/, V/]. 

Lower and upper bounds V 2 /. ., V 2 /j. G R, i = 1, . . . , n and j = 1, . . . , n define an interval 

i j J 

matrix of the type §5§, which is denoted by [V 2 /] = [V 2 /, V 2 /]. Intervals vectors and matrices 
are added component by component. The multiplication of an interval vector or matrix by an 
interval is understood componentwise. Finally, let eW g R™ be defined by = where Sij 
is Kronecker's S, and let Z denote the zero matrix of dimension n x n. 



[a m ,a m ] 
[a m ,a m ] 

[0, max (—a, a) m ] 
\fa G \/\a\ V 7 ^], 
exp(a) G [exp([a])] :— [exp(a), exp(a)], 
ln(a) G [ln([a])] := [lna,lna], 
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2 Numerical calculation of eigenvalue bounds of Hessian 
matrices on hyperrectangles 

In this section we introduce the methods for the calculation of eigenvalue bounds that are 
applied to the collection of test cases in Sect. [3] We give only a short introduction, since these 
methods have been explained in detail elsewhere [2J [6j [3 QUI [12] . 

The compared methods have in common that they are based on a codelist. A codelist results 
if a function tp is broken down into a sequence of elementary unary and binary operations. More 
specifically, let <p : U C K n — > M. denote a twice continuously diffcrcntiablc function. Assume (p 
can be evaluated at an arbitrary point x g U by carrying out a finite sequence of operations of 
the form 

2/i = xi 



Vn+i = $„+i(yi, ■ • ■ ,y n ) / 16 ^ 

Vn+2 = $n+2(yi, ■ • ■ , 2/n, 2/n+l) 

j/ n+t = $„ +t (yi,...,?/„,y n+ i,...,y n+t _i) 

= 2/n+t 

where each fc = l,...,t, represents one of the elementary operations listed in the first 

column of Tab. Q] The codelist lines y k for the example <p(x\, X2, £3) = exp(a;i — 2 
are given in the second column in (|17|) below. The interval extension of a function (p can be 
evaluated by replacing the operations in each line of (|16p by their interval variants listed in 
Fact[T] For the example this results in replacing the yk from the second column by the [yk] from 
the third column of (fl~7|). 

Table 1: Rules for the calculation of yk, [yk], [y'k] an d [y' k '] in the fc-th line of the codelist (fl6|) . 
[yi] refers to the interval gradient of line k with respect to x. The operations shown here have 
been selected more or less arbitrarily to accommodate a reasonably large collection of examples 
ip treated in Sect. |3l The list of can easily be extended [12] . 



op *fc 


Vk 


[yk] 


[y' k ] 


Wl] 


var 


Xk 


[Xk] 




[Z,Z] 


addConst 


Vi + c 


[yi] + [c, c] 


Wi] 


[y'J] 


mulByConst 


cyi 


c[Vi] 


c[y'i] 


cM] 


add 


Vi + Vj 


[yt] + [yj] 


[y'i\ + W 3 ] 


[y'{] + [y'J] 


mul 


ViVj 


[Vi] [Vj] 


h] W + &] M 

r i v r / 1 


[yi] m + [yj] M'] + M [vT + [y'Mf 

r lO >V, r i /r n r nT\ r m\ 



■./<; [Vk] LWJJ L-V-J >- (.•/».. U.V,j U/;. ' I-,'; ' 

^are y| [ % ] 2 2 [ % ] 2 ([yj] [y^] T + [y 3 ] [y'J]) 

c^e yf [ yj f Hy,] 2 ^] HyA(H[y' 3 ][y'j\) T + [y^[yj]) 

powNat yf [y^]™ m [y,]^ 1 ty] to [y.j] m ~ 2 ((to - 1) ([yj] [y'j] T ) + [yj] [y"]) 

sqrt jyj [^~]] 1/ (2 [y k ]) [y>] 1/(2 [y fe ])([y"] + l/(-2 ([$ [y^)) 

exp exp(y,) [expfljy])] [y fc ] [y<] [y fe ] ([yj] [y^-] T + [yV]) 

m in(%) MhM VMWj] i/[%](K] -i/[%] (b'][y'] T )) 
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2.1 Interval Hessians, Hertz and Rohn's method, and Gershgorin's 
circle criterion 



Just as for the calculation of the interval extension of a function, a codelist can be extended 
to calculate gradients Vy? and Hessians V 2 ip and their interval extensions [Vip] and [V 2 <^] by 
combining automatic differentiation (see, e.g., [3[T4]) and interval arithmetics (see, e.g., [13]). 
The required results are summarized in the following lemma, which summarizes results from 
[5]. We recall that Z denotes the zero matrix of dimension n x n. 

Lemma 1 /<5j/ Assume ip is twice continuously differ entiable on U and can be written as a 
codelist. Let B C U be a hyperrectangle. Then, for all x € B, we have <p(x) € [cp], Vtp(x) £ [Vy], 
and W 2 (p(x) £ [V 2 (^], where [ip], [V</s], and [V 2 (/s] are calculated by the following algorithm. 

1. For k = 1, . . . , n, set [y k ] = [x k ,x k ], [y' k ] = [e« e«], and set = [Z, Z\. 

2. For k = n + 1, . . . , n + 1, calculate [yk], [y'k] and [y' k '] according to columns 3—5 of Tab. [H 
respectively. 

3. Set [<p] = [y n+t ], [Vtp] = [y' n+t ], and [V 2 ^] = [y'^ +t ]. 

We refer to a codelist (|16p . that has been extended by additional operations for the calcu- 
lation of interval extensions or derivatives, as an extended codelist for short. The codelist (1161) 
and the extended codelist that results from Lemma Q] are illustrated with an example. 

Example 1 (interval Hessian for exp(xi — 2 ) ) Let B C R 3 be an arbitrary closed 

hyperrectangle and consider ip : B — > R, <p(xi, X2, £3) = exp(xi — 2 ) . Lemma [7] results 

in the following expressions for [yk], [y'k], an d [y'k]> which are first stated in a table for brevity. 
The expressions for yk stated in (|17[) do not result from Lemma[l\ but are given for illustration 
of the codelist Mb}) of ip. 



k 


2/fc 


[yd 


[y'k] 


[y'k] 


1 


Xl 


[xi] 


([1,1], [0,0], [0,0])'^ 


[Z,Z] 


2 




fa] 


([0,0],[1,1],[0,0]) T 


[Z,Z] 


3 


X3 

..'2 


N 
r.. V2 


([0,0],[0,0],[l,l]) r 
nr.. 1 r_ ./ 1 


[Z,Z] 

/r. ./ 1 r../iT 1 r_ . 1 r..//i n 



vl [y 3 ] 3 3[ y3 ] 2 [^] 3[ 2/ 3](2([ 2 / 3 ][y 3 ]) T + N[y 3 ']) , m 

6 -2J/4 -2N -2[y' 4 ] -2 

7 3J/5 3 [2/5] 3 [y> 5 ] 3 [y'£] 

8 2/1 + 2/6 [yi] + [y e ] [y'i] + [y' 6 ] [y'{] + Ml 

9 2/7 + 2/8 [y 7 ] + N [y'r] + [y's] [y"} + M 

10 exp(2/ 9 ) [exp([2/ 9 ])] [y w ][y' 9 \ [VwjM + 

V = 2/io W\ = [2/10] [Vip] = [2/io] [V=V] = [j/i'o] 

TTie codelist for ip of the form I116\) results from rewriting the second column of U7\ l as y\ = ccx, 
2/2 = x 2 , 2/3 = £3, 2/4 = 2/i> - • • > 2/10 = exp(2/ 9 ),<^ = 2/10- ?7ie extended codelist for [y k ] can be 
constructed by carrying out the expressions for [yk], [y'k], o-nd [y k ] and storing the results line 
by line, i.e., 

[2/1] = [xi], [y[] = ([i,i],[o,o],[o,o])TK] = [z,z], 

: : : (18) 

[y w ] = [exp([2/ 9 ])], [y> 10 ] = [y 10 ] [y' 9 ], [y>{ ] = [y 10 ] ([y 9 ] [2/ 9 ] T + M), 
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where the interval Hessian [V 2 </?] reads as [y" ] in the codelist notation. Note that the interme- 
diate interval function values \y^\ and the derivatives [y' k ] are needed to calculate [V 2 ^], while 
the intermediate function values y^ of the original codelist for ip are not. 

After calculating the interval Hessian [V 2 ^] with Lemma [TJ the relaxed problem ^ can 
be solved with a number of methods (see [8] for an overview). As pointed out in Sect. [1] we 
choose Gershgorin's circle criterion for its favorable computational complexity (see Sect. 12 .3[) . 
In addition, we apply Hertz and Rohn's method, because it provides the tight eigenvalue bounds 
that solve ©. The interval variant of Gershgorin's circle criterion and Hertz and Rohn's method 
are summarized in the following two lemmas. 



Lemma 2 (interval Gershgorin [Jllfi]) Let [V 2 y . ., V 2 ^,-], i,j = l,...,n be intervals that 
define a symmetric interval matrix of the form ®. Then 



A = min V ip.. — ri, A = max V 2 <^« + r, (19) 

iS{l, ...,«} M ie{l,...,n} 



where the Gershgorin-radii r» are defined by n — 2j=i j^i max(— V 2 ^ ., W 2 ^^), solve proble 



Lemma 3 (Hertz [7J and Rohn [16j) Let [V 2 y. ., V 2 y>jd, i,j — l,...,n be intervals that 

define a symmetric interval matrix of the form |5|]. Define the matrices S^ k ' G R nx2 for 
k = 1, . . . , n recursively by 

Sik)= {l Sik ^l -l^-l) ' 5(1) = ( X 
Define the symmetric matrices g R™ x " and € W nxn for k = 1, . . . , 2 n ~ 1 according to 

L W = { i = 3 or S W • S W = 1 ^ ^ = | VV, if if = ^ 

lj \ V 2 <Pij otherwise ' y [ V 2 y^ otherwise 

Then 

A = min A min (L< fc >), A= max A max (C/W), (20) 
fee{i,...,2"-i} fce{i,...,2"-i} 

where A m ; n (A) and A max (A) denote the smallest and largest (real) eigenvalue of any symmetric 
real matrix A = A T , respectively, solve problem {Tjp. 

We briefly illustrate Lemmas [2] and [3] by applying them to the sample function from Example 

m 

Example 2 (Gershgorin and Hertz/Rohn applied to exp(xi — 1x\ + Sx^)) Without detailing 
the calculations we claim that substituting B = [—0.3,0.2] x [—0.1,0.6] x [—0.4,0.5] into the 
extended codelist (|18j) yields 



/ [0.298,1.777] [-4.265,0.7109] [0.000,3.999] 
[VV] = [y" ] = [-4.265,0.7109] [-7.109,3.128] [-9.597,1.599] 
\ [0.000,3.999] [-9.597,1.599] [-12.795,24.991] 

Applying Lemma\^ yields the n — 3 Gershgorin radii n = 4.265 + 3.999 = 8.264, r-i — 4.265 + 
9.597 = 13.862 andr% = 3.999 + 9.597= 13.596. Upon substitution into H19\) the spectral bounds 



[Ag] = [A G , A G ] = [-12.795 - 13.596, 24.991 + 13.596] = [-26.391, 38.587] (21) 
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result, where the subscript G is short for Gershgorin. Hertz and Rohn's method requires to 
calculate 2 • 2™~ 1 = 2" = 8 vertex matrices . . . , and U^, . . . , and the sign matrix 
5( 3 ) defined in Lemma\^ Equation \20\) yields 

[Ah] = [A H , Ah] = [A min (£ (2) ), A max ([/( 3 >)] = [-20.597,29.603] (22) 

/ 0.298 0.711 3.999 \ / 1.777 -4.265 3.999 \ 

with = I 0.711 -7.109 -9.597 and = -4.265 3.128 -9.597 , where 
\ 3.999 -9.597 -12.795 / \ 3.999 -9.597 24.991 / 

the subscript H is short for Hertz and Rohn. We only list the matrices and that 
are selected in the minimization and maximization in t20\) and omit the remaining six vertex 
matrices for brevity. 

2.2 Eigenvalue arithmetic 

We summarize the eigenvalue arithmetic in Lemma 0] and Tab. [5] and refer the reader to [TUHT2"] 
for details. LemmaHlimplies that the eigenvalue arithmetic does not require the interval Hessian, 
but interval gradients suffice. The functions A s and A t used in Lemma0]and Tab. are defined 

as 



[A.([o])] 

and 



(23) 



0,^ max (a-, a?) 

i=l 

n 

[A t ([a], lb})} = [-/?,/?] + %M (24) 



where j3 = \J (J2i=i m a x (a 2 , a? ))E™=i max (^i > &i ))■ We refer to [12] for a detailed discussion 
of the meaning of [A 8 ([o])] and [At ([a], [6])]. 

Table 2: Rules for the calculation of [Afc] in the fc-th line of the codelist <jT6j • We assume that 
[yi}, [y'j\ and [yj], [y'j] for all previous lines i < k, j < k have been calculated according to the 
rules from Tab. Q]and can be reused in line k. Rules for yk are repeated here for convenience. 
The functions [A s ]([a]) and [A t ]([a], [b}) are defined in <J23]) and (l24l) . 



op <$>k 


Vk 


[Afe] 


var 


Xk 


[0,0] 


addConst 


Vi + c 




mulByConst 


cVi 


c[Ai] 


add 


Vi + Vj 


M + [A,] 


mul 


ViVj 


M [Ai] + fa] [Xj] + [A t (M, h,;.])] 

r T> , n r m / r / i \ i F"\ T\ ^ 



'J J 



[y k V(2[y k ] [AsM])]-[Xj 



square ^ 2 ([A s ([^])] + [ % ] [A,]) 

cube y] 3[ 2/i ](2[A s ([^])] + [t/ J -][A i ]) 

powNat yf m[yj] m ~ 2 ((m—l)[A s ([y'j])] + [yj] [\j] 

sqrt 0y 1/(2 [ 2/fe ])([A,] + 1/(-2 fe])[A s ([^])]; 

exp exp(j/j) [y/c] ([A s ([^-])] + [Aj]) 

m ln(%) l/[%] ([A,] - l/[ yj ] [A s ([y>])}) 
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Lemma 4 (eigenvalue arithmetic [12]) Assume tp is twice continuously differ entiable on U 
and can be written as a codelist. Let B C U be a hyperrectangle. Then, for all x € B, we have 
tp{x) <E [tp], Vy(x) S [Vy>], and Am € [Am] for all eigenvalues of "V tp(x), where [tp], [Vy>], and 
[A™] are calculated by the following algorithm. 

1. Fork = l,...,n, set [y k ] = [x k ,x k ], [y' k ] = [e<- k \ e^j, and set [A fe ] = [0,0]. 

jS. for = n + 1, . . . , n + £, calculate [yk], [y' k ] and [X k ] according to columns 3 and 4 of 
Tab. [JJ and column 3 of Tab. respectively. 

3. Set [tp] = [y n+t ], [Vtp] = [y' n+t ], and [Am] = [A„ +t ]. 

Lemma HI is illustrated with the sample function from Example Q] and [2] 

Example 3 Le£ -B and p : £> — > K be as in Example [7J Applying Lemma U^to <p results in the 
expressions for [yk], [y' k ], an d [Afc] listed in (|25|l . which we state in a table first for brevity. Note 
that [y k ] and [y' k ] are equal to those in (|17[) . These expressions are repeated here, since the [X k ] 
depend on them. 



k 


[yk] 




[A fc ] 




1 




([l,l],[0,0],[0,0]) r 


[0,0] 




2 


[X2\ 


([0,0],[1,1],[0,0]) T 


[0. 0] 




3 


[X3\ 


([0,0],[0,0],[1,1]) T 


[0,0] 




4 


[V2f 


2 hft] fa£] 


2([A s ([y' 2 ])] + [y 2 


] [A 2 ]) 


5 


[y 3 ] 3 


3N 2 [^] 


3 [i/ 3 ] (2[A s ([y 3 ])] 


+ [ys] [A 3 ]) 


6 


-2 M 


-2M 


-2 [A 4 ] 




7 






3 [A 5 ] 




8 


[vi] + [ye] 


[2/i] + M 


[Ai] + [A 6 ] 




9 


[vt] + [ys] 


M + [ys] 


[A 7 ] + [As] 




10 


[exp([y 9 ])] 


[yio] [y' 9 ] 


[yio]([A s ([^])] + 


[A 9 ]) 




[<p] = [yw] 


[Vy>] = [y 10 ] 


[Am] = [A10] 





(25) 



The extended codelist for [Am] results from evaluating and storing the expressions listed in 
(|25p line by line, i.e., 

[yi] = [xi], [yi] = ([1,1], [0,0], [0,0])^] - [0,0], 

: : : (26) 

[y w ] = [exp([y 9 ])], [y' 10 ] = [y w ][y' 9 ], [A 10 ] = [yio] ([A s ([^])] + [A 9 ]), 

Without detailing the calculations we claim that applying (|26|) to the particular hyperrectangle 
B = [-0.3,0.2] x [-0.1,0.6] x [-0.4,0.5] from Example^ results in 

[Aa] = [Am] = [-19.904,37.004], (27) 

where the subscript A is short for arithmetic. 

By comparing the spectral bounds ([2"Tj) , (|2"2"|) and ([2"7]) , we find the relations A G < A H < A A 
and Ah < Aa < Aq for the discussed example. Note that the lower bound from the eigenvalue 
arithmetic is tighter than the tight bound for the interval Hessian obtained with Hertz and 
Rohn's method. We stress that these relations do not hold in general. It is the very point of 
Section [3] to analyze these relations for a large collection of examples. 
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2.3 Computational complexities 



The discussed methods do not only differ with respect to the tightness of the eigenvalue bounds, 
but also with respect to computational cost. Calculating the interval Hessian matrix with 
forward mode automatic differentiation and applying Hertz and Rohn's method requires 

0{n 2 )N{<p) +0(2 n n 3 ) (28) 

operations [12], where N(<p) denotes the number of operations needed for the evaluation of <p 
at a point. Calculating the interval Hessian and applying Gershgorin's circle criterion takes 

0{n 2 )N{p) +0{n 2 ) (29) 

operations [12]. Note that 0{n 2 ) N(ip) operations are needed for the calculation of the interval 
Hessian in both ([28]) and ([29]) . Calculating eigenvalue bounds with the arithmetic from [12] 
requires 

0(n)N{tp) (30) 

operations [T^]. Due to the 0(2™ n 3 ) term in ([2"5| the computational cost of Hertz and Rohn's 
grows drastically compared to ([29]) and (l30l) . The complexities ([29]) and ([30]) . however, differ 
only by one order of magnitude. We therefore compare the computational effort of these two 
methods more precisely in the present and subsequent section. 

We denote the eigenvalue bounds calculated with the arithmetic proposed here and Ger- 
shgorin's circle criterion [Aa] and [Ag], respectively. The number of operations required to 
calculate [Aa] and [Ag] for a specific function ip are denoted Na(<p) and Nq{<p), respectively. 

Table 3: Number of operations necessary to calculate y k , [yk], [y' k ], l^k] and [y k ] for each type 
of line of a codelist (ITBl . N([y' k ]) denotes the number of operations necessary to compute [y' k ] 
assuming that [yk] is already available. N([Xk}) and N([y k ]) denote the number of operations 
necessary to compute [Afc] and [y"], respectively, assuming [y k ] and [y' k ] are available. Listed 
numbers apply for n > 1. 



op $fc 


N(y k ) 


N([y k ]) 


N([y' k }) 




N([y'k\) 




var 


1 




















addConst 


1 


2 

















mulByConst 


1 


2 


2n 


2 




n(n-\ 


-1) 




add 


1 


2 


2n 


2 




n(n 4 


-1) 




mul 


1 


8 


18 n 


18 n 4 


- 21 


19 n (n 


+ 1) 




oneDver 


1 


2 


8n + 7 


4n + 


26 


14 n (n + 


1) + 


7 


square 


1 


5 


8n + 2 


4n + 


11 


10 n(n 


+ 1) 




cube 


1 


2 


8n + 7 


4n + 


21 


14 n (n + 


1) + 


2 


powNat 


1 


5 


8n + 7 


An + 


2G 


14n (n + 


1) + 


7 


sqrt 


1 


2 


8n + 4 


4n + 


25 


13 n (n + 


1) + 


8 


exp 


1 


2 


8n 


4n4 


- 9 


9n(n- 


f 1) 




In 


1 


2 


8n + 2 


4n + 


21 


13 n (n + 


1) + 


4 



The exact number of operations needed to calculate eigenvalue bounds for a specific function 
can be determined for any of the discussed methods by counting operations in the extended 
codelist of (p. Table [3] lists the exact number of operations needed in each codelist line by line 
type. An operation counted towards N(y k ), N([y k ]), N([y' k ]), N([\ k ]), or N([y k ]) in Tab.[|]may 
either be an addition, multiplication or comparison of two real numbers, or the application of 
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one of the functions oneOver, square, cube, powNat, sqrt, exp or In. Note that this way of 
counting operations is coarse but a standard approach in the field of automatic differentiation 

Before applying Tab. [3] to specific examples in Sect. [31 we derive some general statements. 
From the last two columns of Tab. [3J we infer N([Xk]) < N([y k ]) for all codelist line types 
(assuming n > 1). Since the [y' k '\ are required for the calculation of eigenvalue bounds with 
Gershgorin's circle criterion, N([Xt]) < N([y k ]) for all implies 

N A (tp) < N G (<p) (31) 

for any function ip that can be stated as a codelist with lines of the types from Tab. [3J Fur- 
thermore, inspection of Tab. [3J shows that the eigenvalue arithmetic can be applied at little 
additional computational effort, whenever eigenvalue bounds are calculated by applying Ger- 
shgorin's method to the interval Hessian. This statement holds, since the [y' k ] required for the 
arithmetic are available as an intermediate result to the interval Hessian calculation. More 
specifically, 

t 

AN A (cp)=Y / N([X n+k \) ( 32 ) 
fc=i 

additional operations arc needed to calculate eigenvalue bounds with the arithmetic, if they are 
calculated by applying Gershgorin's circle criterion to the interval Hessian matrix. We infer 
from Tab. [3jthat AN A (<p) as defined in (|32[) amount to 0(n) operations. 

Example 4 ( number of operations for Examples]]^® and\3ty Table^ lists the particular numbers 
of operations necessary to evaluate [yk\, [y' k ], [Xk] and [y' k '] for the sample function (p(x) — 
exp(iri — 2 ) according to the codelist [H\ ). We find N A (ip) — N{[Xa\) = N([X V ]) = 

17 + 73 + 73 = 163 and N G (ip) = N{[V 2 ip]) + N([X G ]) = 17 + 73 + 222 + 12 = 324 7 where 
N([Xg\) = 12. The additional effort for calculating X A given the intermediate results [yk] and 
[y' k ] yields AN A (ip) = 73. 

Table 4: Numbers of operations for the codelist lines of Example [1] 



k 


op *fc 


N{[y k }) 


N([y' k }) 


N([X k ]) 




4 


square 


5 


18 


19 


60 


5 


cube 


2 


23 


29 


86 


6 


mulByConst 


2 


4 


2 


6 


7 


mulByConst 


2 


4 


2 


6 


8 


add 


2 


4 


2 


6 


9 


add 


2 


4 


2 


6 


10 


exp 


2 


16 


17 


54 


E 




17 


73 


73 


222 



3 Benchmark: Arithmetic versus Gershgorin and Hertz 

We apply the eigenvalue arithmetic to a large collection of examples and compare results to 
those obtained by applying Gershgorin's circle criterion and Hertz and Rohn's method to the 
interval Hessian. Sections 13.11 and [3.21 describe the test examples and the scheme of comparison. 
The actual results are summarized in Sect. 13.31 
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3.1 Collection of test cases 



The test cases are extracted from the COCONUT collection of optimization problems )17j . 
We consider all COCONUT problems with 1 < n < 10 variables and extract those cost and 
constraint functions that can be decomposed into the operations listed in Tabs. [T] and [2] respec- 
tively. This results in a set of 1522 sample functions ip : M™ — > K with 1 < n < 10. For each 
ip, we generate 100 random hyperrectangles B C D C R™ in the domain D of tp specified in 
the respective COCONUT problem. Each of the three methods introduced in Lemmas [2] Q] is 
applied to the resulting 1522 • 100 sample problems. 

We omit examples with n = 1, since the three methods yield identical spectral bounds and 
require the same numerical effort. The upper bound n < 10 is arbitrary. The comparison in 
Sect. l3~3l corroborates that the eigenvalue arithmetic benefits more and more from its favorable 
computational complexity as n increases, which was anticipated in the comparison of computa- 
tional complexities in Sect. l2.3l While the eigenvalue arithmetic and Gershgorin's circle criterion 
could be applied well beyond n = 10, it becomes tedious to calculate the exact Hessian matrix 
eigenvalue bounds for comparison, due to the 0(2" n 3 ) complexity of this problem. 

Table 5: Excerpt of the set of examples taken from the COCONUT-benchmark. 

n a ii. ic // function ip 



ex8_l_6-l 2 
ex7_2_6-2 3 
exl4_2_2-6 4 



l i l i l 

(x 1 -4) 2 + (x 2 -4) 2 +0.1 T (i;i-l) 2 + (2;2-l) 2 +0.2 T (x 1 -8) 2 + (x 2 -8y 2 +0:2 

1 - 0.01 f| - 0.01 x 1 - 0.0005 xi x 3 
10.208 - 2 ! 5 9 5 1Q 6 f fi1 2^192^ T^^p— ln(0.316 x x + x 2 ) + x 4 

£3+219.161 xi+0. 192 £2 0.316 £1+3:2 v J 



Table[5]lists three sample functions from the COCONUT collection for illustration. We refer 
to all examples by their COCONUT name, for example ex8_l_6. The suffix -i, as in ex8_l_6-l 
for example, uniquely identifies the function in the respective COCONUT optimization problem, 
where i = 1 corresponds to the objective function and i = 2, . . . , m corresponds to the (i — l)-th 
constraint function^. 

We stress that we use the described set of test functions without further modifications. 
There exist functions in the collection treated here that contain convex terms, or terms for 
which tight convex under- or tight concave overestimators are known (e.g., bilinear, trilinear, 
linear fractional terms) [21 19] . Depending on the application it may be advisable to separate 
these terms from the given function ip, and to calculate Hessian eigenvalue bounds only for the 
remaining terms of p. Here we choose not to apply any preprocessing for the sake of an unbiased 
comparison. 

3.2 Evaluation of results 

We introduce a simple rating scheme that assigns each result to one of a finite set of classes. 
Specifically, we distinguish between the cases listed in Tab. |6l which reflect that a bound from 



3 Ordering is as in the GAMS code provided in the COCONUT library. 
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the eigenvalue arithmetic may be 

(— ) worse than the bounds from the other two methods, 

(o) equal to the one from Gershgorin's method, but equal to or worse than 

the one from Hertz and Rohn's method, 
(+) better than the one from Gershgorin's method, and equal to or worse 

than the one from Hertz and Rohn's method, 
(++) better than the one from Hertz and Rohn's method. 

Note that bounds calculated with Hertz and Rohn's method are never worse than those from 
Gershgorin's circle criterion, since Hertz and Rohn's method provides the tight eigenvalue 
bounds for an interval matrix. Consequently, the bounds from Gershgorin's method do not 
play a role in our definition of the (++) category. Furthermore note that we do not distinguish 
between A G = A H and A G < A H (resp. Ag = Ah and Ag > Ah) in the case A A < A G (resp. 
A A >A G ). 

Table 6: Classes used to aggregate results in Sect. 13.31 Symbols [Aa] = [Aa, Aa], [Ag] = [A g ,Ag], 
and [Ah] = [A h ,Ah] denote the eigenvalue bounds calculated with the eigenvalue arithmetic, 
Gershgorin's circle criterion, and Hertz and Rohn's method, respectively. 



bound 


(-) 


ch 

(°) 


iss 

(+) 


(++) 


upper (Aa) 
lower (A A ) 


Aa > A G > A H 
Aa < Ag ^ Ah 


Aa = Ag > Ah 
Aa = A G < Ah 


Ag > Aa > Ah 
Ag < Aa ^ Ah 


Ag > A H > Aa 
Ag ^ Ah < Aa 



Table [7] lists some numerical results. These examples illustrate that the classes introduced 
in Tab. [5] are meaningful. In particular it is evident that eigenvalue bounds of the same function 
may fall into different classes for different hyperrectangles B. 



Table 7: Illustration of the classes (— ), (o), (+), (++) introduced in Tab.0 



example 


illustrative- 1 


illustrative-2 


<P 


exp(xi — 2 


3^2 + 3 £3) 


Xi 

xi+0.2x'i 


2 — 22 — 

12+0.3^ 




[-0.3,0.2] 


[-0.198,0.177] 


[1.043,1.535] 


[1.5,1.6] 


B [x 2 ,x 2 ] 


[-0.1,0.6] 


[-0.473,0.2] 


[0.6,1.969] 


[0.6,1.1] 


[£3,^3] 


[-0.4,0.5] 


[-0.392,0.39] 


[0.555,0.772] 


[1.0,1.6] 


A [A a ,Aa] 


[-19.904,37.004] 


[-15.767,19.27] 


[-43.934,27.391] 


[-45.014, 17.624] 


G [Ag,Ag] 


[-26.391,38.587] 


[-15.767,18.443] 


[-44.907,27.391] 


[-40.725, 19.507] 


H [Ah, Ah] 


[-20.597,29.603] 


[-12.603,14.278] 


[-34.743,26.399] 


[-33.691,18.897] 


class Aa 


(+) 


(-) 


(°) 


(++) 


Aa 


(++) 


(°) 


(+) 


(-) 



3.3 Results 

Table [8] summarizes the results obtained for the 1522 sample functions. The numbers listed 
in the columns labeled Aa state for how many of 100 randomly generated hyperrectangles the 
lower bounds calculated with the three methods fall into the classes (— ), (o), (+) and (++) 
defined in Tab. [SI The numbers listed in the columns labeled Aa state the corresponding results 
for the upper bounds. 
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Table 8: Summary of results for the 1522 sample functions extracted from the COCONUT 
collection. For each example, lower and upper bounds on Hessian matrix eigenvalues were 
calculated with the three methods introduced in Lemmas [2HI] for 100 random hyperrectangles. 
Numbers state for how many out of the 100 random hyperrectangles the bounds belonged to 
the classes (— ), (o), (+) and (++) defined in Tab. [6l Shaded cells highlight empty classes. 
Horizontal lines divide characteristic groups, e.g. examples with an empty class (H — h) (rank 
212 < r < 854). The averages listed in the last row take all 1522 examples into account, 
including the ones not shown here. 



rank 


example 






benchmark A A 






benchmark Aa 




r 


name 


n 


(-) 


\°) 




(++) 


(-) 


\°) 


\ + ) 


(++) 


1 


box3-l 


3 


2 


50 


32 


16 








2 


98 


2 


box2-l 


3 





56 


26 


18 








6 


94 


3 


cliff-1 


2 


22 


54 


1 


23 





10 


1 


89 


4 


chaconnl-1 


3 


29 


45 





26 





14 





86 


5 


chaconn2-l 


3 


19 


63 





18 





11 





89 


6 


cb3-l 


3 


18 


62 





20 





14 





86 


7 


polak6-l 


5 








100 














100 


8 


polak6-2 


5 








100 














100 


9 


polak6-3 


5 








100 














100 


10 


polak6-4 


5 








100 














100 


11 


growth- 1 


3 








96 


4 








4 


96 


12 


alsotame-1 


2 








100 














100 


13 


vardim-1 


10 











100 








100 





14 


vardim-2 


10 








100 














100 


15 


alsotame-2 


2 





u 


i no 

1UU 








n 
u 


n 


100 




UI (JWIllJ.fc2.Ll ± 


4 


i 

_L 


u 


QQ 

yy 


n 


n 
u 


u 


A 

u 




1 7 


pn c c — X 


2 


Q 
O 


o 


97 


n 


n 

LI 


o 


o 


ion 


18 


v ciii d. e jrm 1 — 1 


10 


75 


n 
u 


ZiJ 





o 


n 
u 


n 


100 


19 


ex8 1 7-1 


5 


99 





1 


o 


o 








100 


20 


hs026-2 


3 





100 

















100 


209 


C> .A. _L ~ _i. / \J 


10 


4 


n 


uo 


I 


25 


n 

u 


7^ 


o 


210 


pxl4 1 7-9 


10 


25 


n 

u 


7^ 
l o 


o 


4 


n 
u 




1 


211 


TinnmcnT't' — 1 


g 


100 


n 

u 


n 

u 


o 


96 


n 


■3 
O 


x 


212 




2 


n 


Q 


100 


n 


n 

u 


o 


100 


n 
u 


213 


ship— 15 


10 


1 





99 




• 


1 





99 


o 


852 


butcher-4 


7 


100 











99 





1 





853 


i5-3 


10 


100 











99 





1 





854 


cohn3-l 


4 


100 











99 





1 





855 


ex4_l_8-l 


2 





100 











100 








856 


sample-3 


4 





100 











100 








1391 


womf let-1 


3 


100 











97 


3 








1392 


reimer5-2 


5 


98 


2 








99 


1 








1393 


reimer5-5 


5 


98 


2 








99 


1 








1394 


ex7_2_9-4 


10 


100 











100 











1395 


ex7_2_9-2 


10 


100 











100 











1518 


cohn2-2 


4 


100 











100 











1519 


cohn2-3 


4 


100 


l> 








100 











1520 


cohn2-4 


4 


100 










100 











1521 


boon-2 


6 


100 











100 











1522 


boon-4 


6 


100 











100 











arithmetic average 


24.17 


60.85 


12.48 


2.50 


23.52 


60.33 


10.63 


5.52 



The examples are ranked in Tab. |5] by, loosely speaking, the quality of the bounds found 
with the eigenvalue arithmetic. More precisely, an example is ranked the higher, the higher the 
sum of the figures in its two (H — h) columns. If this sum is equal for several examples, they are 
sorted according to the sum of the figures in their two (+) columns. Subsequently, the sums of 
the two (o) columns and the two (— ) columns are used for the ranking whenever necessary. 

Table |8] shows the 20 best and 5 worst rated examples and characteristic ranks in between. 
Ranks 213—209, for example, are shown, because they mark the boundary between those if for 
which some eigenvalue bounds still fall into class (++), and the highest ranking examples for 
which class (H — h) no longer occurs. These transitions in the ranking are marked with horizontal 
lines and shaded areas. 

Just as for the illustrative examples given in Tab. [7J all classes (— ), (o), (+), (++) occur 
for the sample functions extracted from the COCONUT collection. In particular there exist 
cases for which the eigenvalue arithmetic provides tighter bounds than the tight bounds for the 
interval Hessian. An analysis of the data given in Tab. [5] reveals that 2.50% and 5.52% of the 
examples belong to the (++) class for the lower bound and upper eigenvalue bound, respectively. 
Furthermore, in 12.48% of the cases the eigenvalue arithmetic provides tighter lower bounds than 
Gershgorin's circle criterion applied to the interval Hessian matrix. In 10.63% of the cases the 
upper bound from the eigenvalue arithmetic is tighter than the upper Gershgorin bound. In 
60.85% (60.33%) of the cases the lower (upper) bounds from the eigenvalue arithmetic and those 
from Gershgorin's circle criterion are equal. We stress, however, that the Gershgorin bounds 
outperform those of the eigenvalue arithmetic in 24.17% (lower bound) and 23.52% (upper 
bound) of the cases, respectively. Overall, there exist 40 examples for which the eigenvalue 
arithmetic provides better bounds than Gershgorin's circle criterion for all random boxes (e.g. 
7 < r < 15 or r = 212). On the other hand, there exist 129 examples for which the arithmetic 
results in less tight spectral bounds for all random boxes (r > 1394). 

For 854 out of the 1522 examples (i.e., 56.11%) the eigenvalue arithmetic provides tighter 
bounds than Gershgorin for at least one of the random boxes. Finally, we note that the ranking 
r does not correlate with the dimension n, i.e., we find both low and high values of n in any 
part of the ranking shown in Tab. [8] The dependency on n is analyzed in more detail at the 
end of this section with Tab. [TUJ 

We discussed in Sect. 12.31 that the compared methods do not only differ with respect to the 
tightness of eigenvalue bounds, but also with respect to their computational complexity. Table 
[9] lists the operation numbers for the examples from Tab. [8] All figures in Tab. [9] are based on 
the total number of operations needed for the respective method. Specifically, Na(<p) denotes 
the total number of operations for calculating [Aa] with the eigenvalue arithmetic, including 
the operations for the intermediate results [yk] and [y' k ]. Ng(<p) denotes the total number of 
operations for calculating [Xq] with Gershgorin's circle criterion, including the operations for the 
intermediate results [yk], [y' k ] and [y k ]. We also list AA?a(</?) defined in (f32|) . i.e. the additional 
effort to calculate [Aa], if [Ag] and its intermediate results [yk] and [y' k ] have been determined. 
As predicted by relation ([3T]) . the eigenvalue arithmetic always requires fewer operations than 
the interval variant of Gershgorin's circle criterion. On average the computational effort for 
the eigenvalue arithmetic amounts to 37.40% of that of applying Gershgorin's circle criterion 
to the interval Hessian, where this figure ranges from 10.45% (example r = 18 in Tab. [9]) to 
60.06% (example r — 161). The average additional effort AAa(v?) for the eigenvalue arithmetic 
amounts to 17.35%, with a minimum and maximum of 3.26% (example r = 14) and 29.96% 
(example r = 838). 

We stress that N\(ip), Nq(<p), AAa(v) do not depend on the particular hyperrectangle B, 
but can be determined for any function tp before eigenvalue bounds are actually calculated. It 
may therefore be an option to determine these operation counts beforehand and to decide which 
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Table 9: Computational effort for the evaluation of eigenvalue bounds for the examples taken 
from the COCONUT-benchmark. Na{<p) and Nc(ip) denote the total number of operations 
necessary to calculate [Aa] and [Aq], respectively. ANa(<p) denotes the additional number of 
operations needed to calculate [Aa] assuming the Gershgorin bounds have already been com- 
puted. The examples are listed in the same order as in Tab. HI Shaded cells highlight minimal 
and maximal values for the ratios jvcty) and ^^^y , respectively. The averages stated in the 
last row take all 1522 examples into account, including those not shown here. 





example 




abs 


. complexity 




rel. complexity (%) 


r 


name 


n 


TV T f \ 

N A ((p) 


JVgO) AiV A (y) 


N A (<p) 
N G (tp) 


N G (<p) 


1 


box3-l 


3 


2270 


5798 


808 


Oil . 10 


1 Q 0/1 

io.y4 


2 


box2-l 


3 


2292 


5820 


808 


"30. 18 
oy.ao 


1 1 88 
lo.oo 


3 


clif f-1 


2 


159 


289 


56 


55.02 


19.38 


4 


chaconnl-1 


3 


87 


242 


29 


35.95 


11.98 


5 


chaconn2-l 


3 


87 


242 


29 


35.95 


11.98 


G 


cb3-l 


3 


87 


242 


29 


35.95 


11.98 


7 


polak6-l 


5 


1302 


5526 


456 


23.56 


8.25 


8 


polak6-2 


5 


1300 


5524 


456 


23.53 


8.25 


9 


polak6-3 


5 


1302 


5526 


456 


23.56 


8.25 


10 


polak6-4 


5 


1302 


5526 


456 


23.56 


8.25 


11 


growth- 1 


3 


3470 


8132 


1546 


42.67 


19.01 


12 


alsotame-1 


2 


59 


120 


23 


49.17 


19.17 


13 


vardim-1 


10 


2828 


21232 


721 


13.32 


3.40 


14 


vardim-2 


10 


3118 


22818 


745 


13.66 


3.26 


15 


alsotame-2 


2 


61 


122 


23 


50.00 


18.85 


1G 


brownden-1 


4 


5696 


18610 


1896 


30.61 


10.19 


17 


price-1 


2 


423 


769 


192 


55.01 


24.97 


18 


vanderml-10 


10 


18188 


174048 


6618 


10.45 


3.80 


19 


ex8_l_7-l 


5 


685 


2920 


229 


23.46 


7.84 


20 


hs026-2 


3 


198 


530 


75 


37.36 


14.15 


161 


gold-1 


2 


961 


1600 


447 


60.06 


27.94 




838 


sendra-1 


2 


599 


1038 


311 


57.71 


29.96 


1518 


corm2-2 


4 


3132 


9296 


1514 


33.69 


16.29 


1519 


corm2-3 


4 


3132 


9296 


1514 


33.69 


16.29 


1520 


cohn2-4 


4 


5067 


14578 


2463 


34.76 


16.90 


1521 


boon-2 


6 


718 


3298 


350 


21.77 


10.61 


1522 


boon-4 


6 


718 


3298 


350 


21.77 


10.61 


arithmetic average 


irrelevant 


37.40 


17.35 
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method to use. This may be an option in applications in which eigenvalue bounds need to be 
calculated for the same function ip for many B such as branch-and-bound global optimization. 

While we did not recognize a dependency of the tightness of the bounds on n, the ratios 
Na{(p)/Ng((p) and AJVa(^)/JVg(p) clearly depend on n. This was anticipated in the discussion 
of the complexity classes in Sect. 12.31 Table [TOl shows that the relative number of operations 
for the eigenvalue arithmetic 7Va(<^)/^Vg( ( / 3 ) improves from about 54% for n = 2 to about 
14% for n = 10. Similarly, the additional effort for the eigenvalue arithmetic decreases from 
about 24% for n — 2 to about 6% for n — 10. Note that examples with n — 1 would yield 
N A (ip)/N G (<p) = 1 = 100%. 

Finally, we note that the large number of examples ip for n — 3 results from the COCONUT 
optimization problem oet2. We did not omit any of these ip in order not to introduce bias. 



Table 10: Aggregation of results by dimension n. 



n 


num. of 

examples 


(-) 


mean: 

(°) 


(%) 
(+) 


(++) 


mean: 

N A (<p) 
N G ( V ) 


(%) 

AJVa(v) 
AT g O) 


std: 

iv A (y) 
Na(ip) 


(%) 

AN A ( V ) 
N Q (ip) 


2 


62 


57.89 


15.47 


14.68 


11.97 


54.16 


24.41 


3.01 


3.17 


3 


1078 


10.88 


79.32 


8.79 


1.01 


41.70 


19.81 


1.04 


1.21 


4 


67 


61.29 


19.34 


8.45 


10.92 




2.76 


2.42 


5 


88 


56.86 


15.85 


12.48 


14.81 




3.14 


2.13 


6 


95 


35.05 


14.21 


36.88 


13.86 


23.08 


8.97 


2.12 


1.47 


7 


27 


65.81 


34.17 


0.02 


0.00 


18.98 


7.44 


1.76 


1.60 


8 


15 


94.23 


4.50 


1.27 


0.00 


17.85 


7.83 


0.99 


0.82 


9 


24 


65.60 


4.21 


18.71 


11.48 


14.48 


6.15 


2.60 


1.80 


10 


66 


57.06 


9.34 


23.74 


9.86 


14.11 


5.90 


2.22 


1.72 


all 


1522 


23.84 


60.59 


11.55 


4.01 


37.40 


17.35 


9.56 


5.17 



4 Conclusion and Outlook 

Our numerical experiments corroborate that the eigenvalue arithmetic always requires fewer op- 
erations than Gershgorin's circle criterion. While this result has been established by comparing 
the complexity classes of the two methods (see Sect. l2.3l and [12 ), it was analyzed quantitatively 
with a large set of examples here for the first time. Specifically, 10.45% (n — 10) to 60.06% 
(n = 2) of the number of operations of the Gershgorin based approach are necessary for the 
eigenvalue method. The average over all examples for all n amounts to 37.40%. As anticipated 
in the complexity analysis in Sect. 12. 3[ the eigenvalue method benefits from its favorable com- 
plexity as n increases (see Tab. [5] for details). We recall that a comparison to the computational 
effort of Hertz and Rohn's method is not reasonable, since Hertz and Rohn's method belongs 
to a very different complexity class (see Sect. 12. 3| ). 

Gershgorin's circle criterion provides tighter lower (upper) bounds in 24.17% (23.52%) of 
the examples. In 60.85% (60.33%) of the cases the lower (upper) bounds from both methods 
are equal. In 12.48% (10.63%) of the examples the eigenvalue arithmetic results in tighter lower 
(upper) bounds than the Gershgorin based approach. Finally, our tests reveal that the number 
of cases in which the eigenvalue arithmetic results in tighter bounds than the tight bounds of 
the interval Hessian, which are obtained with Hertz and Rohn's method, is small (2.50% and 
5.52% for lower and upper bounds, respectively). On the other hand, these figures indicate that 
these cases are not anecdotal or constructed, but they appear in global optimization problems. 
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The eigenvalue arithmetic provides a tighter lower or upper bound than Gershgorin's circle 
criterion for at least one random box in 56.11% of the examples, where these occurencies are not 
correlated with n. This figure suggests to combine the two methods. We claim the eigenvalue 
arithmetic can be applied at an attractive additional cost for, say, n > 5, whenever the Gcrsh- 
gorin bounds have already been calculated, since both methods involve the same intermediate 
quantities ([yk] and [y' k ], see Sect. [2]). Specifically, the additional effort for applying the eigen- 
value method after the intermediate quantities have been calculated in the Gershgorin based 
approach ranges from 24% for n = 2 to about 6% for n — 10 (see Tab. [TO]). This figure decreases 
for increasing n as anticipated from the abstract complexity analysis in Sect. 12.31 Note that 
this combination of Gershgorin's circle criterion and the eigenvalue method will provide tighter 
bounds than Hertz and Rohn's method for the interval Hessian whenever the eigenvalue method 
does. 
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